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I. INTRODUCTION 



Stochastic perturbations are typical for any physical realization of a given dynamical system. Also round-off 
errors, inevitable in numerical investigation of any dynamics, may be considered as a random noise. Quantitative 
characterization of dynamical systems with external stochastic noise is a subject of several recent studies . On 
the other hand, the influence of noise on various low dimensional dynamical systems and the properties of random 
dynamical systems have been extensively studied for many years |3[-|lO||- 

Consider a discrete dynamical system generated hy f : X ^ X, where X is a subset of R'*, in the presence of an 
additive noise 



Xn+l = f{Xn)+S.n, (1) 

where ^1,^2, • • ■ are independent random vectors fulfilling (^„) = and (^n^m) = cr^(5mn- The case with vanishing 
noise strength cr ^ will be called the deterministic limit of the model. Properties of such stochastic systems have 
recently been analyzed by means of the periodic orbit theory Jill . Convergence of invariant measures of the noisy 
system in the deterministic limit has been broadly discussed in the mathematical literature (see for instanc e [p^ pO| ) . 

A dynamical system generated by / is called chaotic if its Kolmogorov-Sinai (KS) entropy is positive Such 
a definition is not applicable for stochastic systems, characterized by infinite entropy. In this case the partition 
dependent entropy diverges if the partition A of the space X is made increasingly finer. 

In this paper we propose a generalization of the KS-entropy for systems with additive noise (^. Since entropy 
diverges also for the pure noise (with the trivial deterministic limit f{x) = I{x) = x, for x £ A"), we study the 
difference between the total entropy of the system with noise and the entropy of the noise itself. Firstly, we set the 
partition fixed, and then we take the supremum over all finite partitions with regular cell boundaries. In this way 
our definition resembles the coherent states dynamical entropy, two of us proposed several years ago ]2^-p^ as a 
measure of quantum chaos. The entropy of the noise, discussed in this paper, plays the role of entropy of quantum 
measurement, connected with the overlap of coherent states and linked to the Heisenberg uncertainty relation. 

Even though our definition is suitable for d-dimensional systems with an arbitrary additive noise, we demonstrate 
its usefulness on simple one-dimensional systems. We choose a specific kind of distribution defining the noise, which 
can be expanded in a finite basis of N functions in both variables x and y. This condition allows us to express the 
n-steps probabilities, required to compute the entropy, as a product of certain matrices. Moreover, we represent the 
Frobenius-Perron operator of the system with noise by an (N + 1) x {N + 1) matrix, and obtain its spectrum by 
numerical diagonalization. The deterministic limit a ~* requires N — > 00, which resembles the classical limit of 
quantum mechanics. 

This paper is organized as follows. In Sect. II the dynamical entropy for noisy systems is defined and some of their 
properties are analyzed. One dimensional systems with expandable noise and their invariant measures are analyzed in 
Sect. Ill, while different methods of computing the entropy are presented in Sect. IV. The entropy for some exemplary 
systems with noise (Renyi map, logistic map) is studied in Sect. V. The paper is concluded by Sect. VI, while an 
illustrative iterated function system, used for computation of the entropy of noise, is provided in Appendix A. 



II. DYNAMICAL ENTROPY FOR SYSTEMS WITH NOISE 



A. Dynamical entropy for deterministic systems 

Let us consider a partition A — {Ei, . .. ,Ek} of X into k disjoint cells. The partition generates the symbolic 
dynamics in the A;-symbol code space. Every n-steps trajectory can be represented by a string of n symbols, v = 
{io, . . . , in-i}, where each letter ij denotes one of the k cells. Assuming that initial conditions are taken from X with 
the distribution fif invariant with respect to the map /, let us denote by fio,- -,»n-i probability that the trajectory 
of the system can be encoded by a given string of symbols, i.e., 

R,,...,r^_, = ^if {{x : X e E,„, fix) e E,,, . . . , r~\x) e E,,^_,}) (2) 

The partial entropies Hn are given by the sum over all fc" strings of length n 

k 

while the dynamical entropy of the system f with respect to the partition A reads 
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H{f;A) := lim -H^. (4) 

n— >oo 77, 



The above sequence is decreasing and the quantity 

k 



Hi^-J2fif{E.,)\n[fif{E,)], (5) 

i=l 

which depends on / only via /x/, is just the entropy of the partition A. We denote it by Hj\^{^f). The KS-entropy of 
the system / is defined by the supremum over aU possible partitions A |2l| 

HKs{f)-^supH{f;A). (6) 

A 

A partition for which the supremum is achieved is called a generating partition. Knowledge of a fc^-elements generating 
partition for a given map allows one to represent the time evolution of the system in /cg-letters symbolic dynamics and 
to find the upper bound for the KS-entropy: HksH) ^ In/cg. In the general case it is difficult to find a generating 
partition and one usually performs another limit, tending to zero with the diameter of the largest cell of a partition, 
which implies the limit k ^ oo. We shall denote this limit by ^ | 0. 

B. Entropy for systems with stochastic perturbation 

For simplicity we consider one-dimensional case taking X = [0, 1] , imposing periodic boundary conditions and joining 
the interval into a circle. We denote the Lebesgue measure on X by m, setting dx — dm{x) (clearly m{X) = 1). The 
noisy system introduced in ([^) will be denoted by fa- From now on we assume that all the random vectors (n S N) 
in have the same distribution with the density Va- Then the probability density of transition from x to y under 
the combined action of the deterministic map / and the noise is given by Va{f{x),y) = Vaif{x) — y), where x, y G X 
and the difference is taken modi. In the pure noise case (/ = /) this density depends only on the length of the jump 
and equals to Pa{x, y) = Va{x — y). 

We assume that fa has a unique invariant measure /i , which is absolutely continuous with respect to the Lebesgue 
measure m (i.e. it has a density Clearly, /i/^ = m, and so pi^ = 1. Moreover, we assume that the measure /^/^ 

tends weakly to fj.f respectively, for cr — > 0, where /i/ is some invariant measure for the deterministic system /. In 
Sect. IIIB we discuss the situation, where the above assumptions are fulfilled. 

Now, let us fix a partition A of X. We define the total entropy Htotifa', A) of the noisy system fa by formulae 
(^ and (^), analogously to the deterministic case. Note, however, that in this case the initial conditions should be 
taken from X with the measure /i/^. As we shall see below this entropy grows unboundedly with k. Hence, we can 
not define partition independent entropy of the noisy system using formula (^), as the supremum in (^ is equal to 
the infinity. On the other hand, there are two kinds of randomness in our model: the first is connected with the 
deterministic dynamics; the second comes from the stochastic perturbation. Accordingly, we split the total partition 
dependent entropy Htot of a noisy system fa given by (|3|) and (^ into two components: the noise entropy and the 
dynamical entropy. The latter quantity characterizes the underlying dynamics fa and is defined by 

Hdynifa', A) := Hfotifa', A) — Hnoise{cr, A) , (7) 

where the entropy of the noise Hnoise (f. A) reads 

Hno,se{cr, A) = Htot{Ia;A), (8) 

and la is a stochastic system given by (^ with f — I {pure noise). Although the both quantities Htot and Hnoise 
may diverge in the limit of fine partition A i {k oo) for a nonzero noise strength, one can make their difference 
Hdyn bounded, taking an appropriate sequence of partitions, as we shall see in the next subsection. 

In order to keep away from the ambiguity in the choice of a partition, we eventually define the dynamical entropy 
of fa as 

Hdynifa) sup Hdynifa; A), (9) 
A 

the supremum being taken over all finite partitions A — {Ei, . . . , Ek} such that m(i?i) — 1/k and midEi) = for 
each i = 1,. ..,k, k G N. We will call such partitions uniform. The restriction to uniform partitions is necessary. 
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since otherwise we may encounter various "pathologies" in the deterministic limit [|oj. Note, that the uniformity 
assumption may be omitted in the case when all the measures /i/^ , iJ,f, and m coincide. 

It seems that in many cases the entropy of the noise (||) tends to zero in the deterministic limit a ^ 0, and the 
dynamical entropy of converges to the KS-entropy of the corresponding deterministic system / (for partial results 
in this direction see |15| and [E2[, for numerical evidence see Sect. VD). 



C. Boltzmann— Gibbs entropy and bounds for dynamical entropy 

In this section we discuss the behavior of the dynamical entropy in the another limit, A i (k ^ oo). 
Next, we introduce the Boltzmann-Gibbs (BG) entropy of the noise: 

ffBG(^) := - / dM/. {^) [ dyPAx - y) InVA^ - v) = - [ d^V^O ^^V^O ■ (10) 
Jx JX Jx 

For interpretation and generalizations of this quantity, sometimes called continuous entropy, consult the monographs 
of Guia§u 1^ , Martin and England , Jumarie |28| , or Kapur . In the simplest case of the rectangular noise 
given by Vtix) := Q{x - b/2)Q{x + h/2)/b, for 1 > 6 > and x & X, the BG-entropy is equal to In b. Note, that 
this quantity vanishes for the noise uniformly spread over the entire space {h = 1), becomes negative for 6 < 1 and 
diverges to minus infinity in the deterministic limit 6 — > 0. 

For the system combining the deterministic evolution / and the stochastic perturbation, the probability density 
of transition from x to y during one time step is given by Vaifix), y) = Vaifix) — y) for x,y £ X. The BG-entropy 
for this system can be defined as 

HbgU.)^-! dfifA^) [ dyrM{x),y)^^'PM{x),y)- (n) 

Jx Jx 



Due to the homogeneity of the noise and due to the periodic boundary conditions the integral over y in (^d|) does not 
depend on x. Therefore for any system / perturbed by a nonzero noise (cr > 0) one obtains 

HBGifa) = HBGi<j) = - I dyV„{y)\nV„{y) . (12) 

Jx 

Applying this equality and using the same methods as in p4| ] we can prove that the total entropy fulfills the following 
inequalities (the first inequality can be deduced from the lower bound for the variation of information obtained in 



Theorem 2.3 from |26|; the second inequality comes from the definition) 

HBG{a)+\nk < Htot{U,A) < HAi^f^). (13) 

For / = / we get 

Hbg{(t) + In fc < Hno^se{<J, A) < In k . (14) 

Hence, both the total entropy Htotifu-, A) and the noise entropy Hnoitseic^, A) diverges logarithmically in the limit 
^ I 0. Let us now study how does the dynamical entropy, which is the difference of these quantities, depend on the 
partition A. 

If the partition A = {X} consists of one cell only {k — 1), we have Htot{fa,A) ~ Hnoise{cr,A) = = 0. Thus, 
the dynamical entropy with respect to this trivial partition equals zero for any system, which guarantees that the 
dynamical entropy given by the supremum over all partitions (|9|) is non-negative. 

Let us now investigate the behavior of the total entropy in the opposite case A i (k oo) for a non-zero noise 
strength a. Performing the time limit (^) we find, as in |Q, that for very fine partitions the total entropy of the 
system is given approximately by the sum of the Boltzmann-Gibbs entropy and the entropy of the partition (this 
statement is again based on the Theorem 2.3 from |p^ ) 

A 1 

Htot{fa,A) « i/BG(<T)+lnfc. (15) 

Observe that due to the property ( p^ ) the right-hand side does not depend on the dynamical system / and the 
approximate equality ( p5| ) holds also for the entropy of the noise Hnoise (c. A) . Therefore dynamical entropy tends to 
zero for both limiting cases 
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Hdyn{faAX}) = (for fc - 1) (16) 

\ujiHdyn{f,,A)^0 (for fc^oo). (17) 

Let us now discuss, what are the minimal and maximal dynamical entropies admissible for a certain kind of stochastic 
noise. From (|l|) and (|l|) we get 

HBG{<y) <Hdyn{U,A) < -HBG{'y)-lnk + HAO^u) < -Hbg{^)- (18) 

Thus the dynamical entropy is bounded from above by ~Hbg{'^)- Combining this with (^7|) one obtain 

< Hdynih) < -Hbg{(t) (19) 

This relation provides a valuable interpretation of the Boltzmann-Gibbs entropy. This quantity, determined by the 
given probability distribution of the noise , tells us whether the character of the dynamics of a specific deterministic 
system / can be resolved under the influence of this noise. For example, the rectangular noise of width h = 1 may 
be called disruptive, since the corresponding BG-entropy is equal to zero, and consequently Hdynifa-, A) = for 
every uniform partition A. Under the influence of such a noise we have no information, whatsoever, concerning the 
underlying dynamics /. Furthermore, it is unlikely to distinguish between two systems, both having KS-entropies 
larger than —Hbg{<^) of the noise present. 

Evidently, in the deterministic limit the maximal entropy tends to infinity. On the other hand, in this case, one 
obtains in ( p^ ) the KS-entropy. This apparent paradox consists in the order of the two limits: the number of cells in 
coarse-graining to infinity and the noise strength to zero. These two limits do not commute. 

Note that several authors proposed different approaches to the notion of dynamical entropy of noisy system. Crutch- 
field and Packard |]30|] introduced the excess entropy to analyze the difference between partial entropies of a noisy 
system and the corresponding deterministic system, and investigated its dependence on the noise strength a and the 
number of the time steps n. 

In order to avoid problems with the unbounded growth of the total entropy for sufficiently fine partitions Gaspard 
and Wang studied e-entropy |3l|, where the supremum is taken only over the class of the partitions, for which the 
minimal diameter of a cell is larger than e. This quantity can be numerically approximated by the algorithm of Cohen 
and Procaccia ||25[| . The e-entropy diverges logarithmically in the limit e — > 0; the character of this divergence may 
be used to classify various kinds of random processes . 

The dependence of the dynamical entropy on time yields another interesting problem. For discrete deterministic 
systems the KS-entropy is additive in time: HKs{f^) — THkbH)- On the other hand it follows from ([l9| ) that the 
dynamical entropy of a noisy system fulfills Hdyn{fJ) < —Hbg{'^), for each time T. Thus for a nonzero a the ratio 
Hdyn{fJ)/T tends to zero in the hmit T — > oo, while for the deterministic dynamics [HKs{f'^)]/T = HKs{f)- The 
symbol /J represents the same deterministic system /, subjected to the stochastic perturbation only once for T time 
steps. The related issue has been recently raised by Fox [|3| in the context of deterministic evolution of Gaussian 
densities. The discontinuity of Hdyn{fJ)/T in the limit cr ^ is a consequence of the fact that the another limits: 
time to infinity and noise strength to zero do not commute. 

In some sense this resembles the noncommutativity of the limits time oo and /i — > in quantum mechanics, 
crucial for investigation of the so-called quantum chaos (see e.g. |^,^). Continuing this analogy even further, the 
entropy of noise corresponds to the entropy of quantum measurement p^j2^ , while the Boltzmann-Gibbs entropy 
Hbg plays the role of the Wehrl entropy js^, recently used by two of us (WS, KZ) to estimate the coherent states 
dynamical entropy p^ . 



D. Systems with rectangular noise 

We now discuss the computation of the entropy of noise for the rectangular noise Vb (see Sect. IIC), with the periodic 
boundary conditions imposed. Computation of the transition probabilities in (^ reduces to simple convolutions of 
the rectangular noise and is straightforward for the first few time steps. For larger n the calculations become tedious, 
and the convergence in the definition of entropy (^) is rather slow (not faster than 1/n). It is hence advantageous to 
consider the sequence of relative entropies which converge much faster to the same limit H{f;A) [ pO|j25| ]. For some 
systems the exponential convergence of this quantity was reported ]37|-|39|] . 

In our analytical and numerical computations we used relative entropies Gn- In all of the cases studied, the term 
Gy gives the entropy lim„^oo Gn with a relative error smaller than 10^^. For a partition consisting of two equal cells 
(fc = 2) and the rectangular noise Pb we obtained an explicit expression for 6*4, as a function of the noise width b. 
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Analytical result obtained in pj| | are too lengthy to reproduce here gives an approximation of the entropy of noise 
with precision 10~*. 

We analyzed the dynamical entropy of the Renyi map fs{x) = [sx] mod i (with integer parameter s) subjected to 
the rectangular noise. Independently of the noise strength, the uniform distribution remains the invariant density of 
this system. For a large noise 6 1 the dynamical entropy is close to zero, since the difference between the noise and 
the system with noise is hardly perceptible. The dynamical entropy grows with the decreasing noise width b and in 
the deterministic case seems to tend to the KS-entropy of the Renyi map Rxsifs) = Ins. For more involved systems 
the computation of the dynamical entropy becomes rather difficult even for this simple rectangular noise. In order 
to avoid calculating fc" different integrals in (||), in the subsequent section we introduce the class of noises for which 
computing of probabilities Pio,- -,in-i ^O'' dynamical system reduces to multiplication of matrices. 



III. SYSTEMS WITH SMOOTH NOISE OF DISCRETE STRENGTHS 



A. Model distribution of noise 



In this section we define the particular discrete family of the probability distributions P/v representing the noise 
and study properties of dynamical system subjected to this noise. As above, we consider one-dimensional space 
X = [0, 1) and impose periodic boundary conditions. We shall look for a kernel Vix^y) homogeneous, periodic, and 
being decomposable in a finite basis 

V{x,y) = r{x-y)^V{0, 
V{x, y) = V{x mod 1, y mod 1), 

N 

r{x,y)=Y, AirUr{x)vi{y), (20) 

Lr=0 

for G M, where A — {Air)i^r=o....,N is a real matrix of expansion coefficients. We assume that the functions 
Ur, r — 0, . . . , N and vi; I = 0, . . . , N are continuous in X ~ [0, 1) and linearly independent. Consequently, we can 
uniquely express / = 1 as their linear combinations. Both sets of base functions form an (N + l)-dimensional Hilbert 
space. The last property in ( pO| ) is necessary in order to proceed with the matrix method of computation of the 
probabilities in (m. 

All these conditions are satisfied by the trigonometric noise 

^'^(O -Cncos^«), (21) 

where N is even (A'' = 0, 2, . . .). The normalization constant Cn can be expressed in terms of the Euler beta function 
B{a, b) or the double factorial 

TT A^!! 

= 5(^,1) = ^^^^ 

We use basis functions given by 

Ur{x) = cos^ (ttx) sin^^^ (nx), 

viix) = cos'(^2/)sin^-'(7ry), (23) 

where x G X and r,l = 0, . . . , N. We do not require their orthonormality. Expanding cosine as a sum to the A^-th 
power in ( pl| ) we find that the (A^ + 1) x (A^ + 1) matrix A, defined in ([20|), is diagonal for this noise 

Alr^Cf,(^^^yir. (24) 

The parameter A^ controls the strength of the noise measured by its variance 

= :^*'(^ + 1) = 7^ I >: (25) 



\k=iN/2) + l I 
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where stands for the derivative of the digamma function 141 
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FIG. 1. Probability density of the noise P{x) for iV = 10, 20, 50 and 100. 




0.4 0.6 
X 

FIG. 2. Transition kernel 'Pjv(/(k), for the logistic map f{x) — 4x{l — x) with the noise characterized by A'^ = 20. Darker 
colour denotes higher value of the kernel according to the attached scale. The variable x is periodic; x = x mod i- 

Fig. 1^ presents the densities of the noise for N = 10, 20, 50 and 100. The deterministic limit is obtained by letting 
A'' tend to infinity. Since N determines the size of the Hilbert space, in which the evolution of the densities takes 
place, it can be compared with the quantum number j 1/h used in quantum mechanics. Note that for every value 
of the parameter N the probability function Vn{x) > for x ^ 1/2 (mod 1), so the analyzed perturbation is not 
local in the sense of Blank ||l^ . 

It is worthwhile to mention that the properties (EOh are preserved for the kernel "Pat (/(a;), y) describing the dynamics 
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of the system with noise (|l|). The expansion matrix A is the same, if one use the modified basis functions defined by 
Mfe(x) := Uk{f{x)), for x G X, which exphcitly depend on the deterministic dynamics /. To iUustrate some features 
of our model we plot in Fig. || the transition kernel VNif{x),y) for the logistic map perturbed by the noise defined 
in ^ with N = 20. 




FIG. 3. Invariant density for the logistic map f(x) = 4a;(l — x) subjected to the trigonometric noise (A'^ = 20): solid line 
represents the leading eigenvector of the matrix D; histogram is obtained by iteration of one million of initial points by the 
noisy map. 




FIG. 4. Invariant densities of the logistic map for parameters of the trigonometric noise A'^ = 10, 20, 50 and 100, together 
with the deterministic limit — > oo. 
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B. Invariant measure for systems with stochastic perturbation 



The density of the invariant measure pp of the system (|l|) is given as the eigenstate of the Frobenius-Perron operator 
Mp corresponding to the largest eigenvalue equal 1. For the deterministic system, the invariant density p fulfills the 
formal equation 

p{y)= [ 5{f{x)-v)p{x)dx. (26) 
Jo 

In the presence of a stochastic perturbation this equation becomes 

Pp{y) = {Mp{pp))[y)= ( f V{x\y)5{f{x)-x')pp{x)dx'dx^ ( V{f{x),y)pp{x)dx , (27) 
Jo Jo Jo 

where Mp is the Frobenius-Perron (FP) operator connected with the noisy system (0). Let us assume that the 
kernel satisfies the conditions ( pO| ) listed in the preceding subsection and it can be expanded as 'P{f{x),y) — 
Y.^r=o^irUr{f{x))vi{y). Then we have 

.l N N 1 

Mp{p){y) = / ^AirUrif{x))vi{y)p{x)dx^^Air[ Ur{f{x))p{x)dx]vi{y) 



iV 1 

'Ur{f{x))p{x)dx]iriy) 

r=0 



for y ^ X, where iir = X];^o^'»'^'- Thus, any initial density is projected by the FP~operator Mp into the vector 
space spanned by the functions Vr'yT = 0,. . .jN, and so its image may be expanded in the basis {vi}i=o....,N ■ This 
statement concerns also the invariant density pp. Expanding pp 

N 

pp = J2qiP)iVL (29) 

1=0 

with unknown coefficients q{P)i and inserting this into ( p8| ) we obtain the eigcncquation for the vector of the coefficients 
g(P) = {(?(P)o,...,g(P)^} 

qiP) - DqiP). (30) 

The FP-operator is represented by the {N + 1) dimensional matrix D = BA, where A is given by (p^), and the entries 
of the matrix B are given by 

Brrn = / Ur{f {x))Vmix)dx . (31) 

Jo 

for n,m = 0, . . . , N . Observe that A does not depend on the deterministic dynamics /, while B depends on the noise 
via the basis functions u and v. Furthermore, note that 

1 

Ur{f {x))Vyn{x)dx (32) 

Jo 

for r, m = 0, . . . , iV, and 

N N 

MpC^ q{P)My)) = (33) 

/=0 /=0 

for each vector q{P) = {q{P)Q, . . . , q{P)j^} € It follows from ( p8|) and ( ^3| ) that there is a one-to-one correspon- 

dence between the eigenvectors of D and the eigenfunctions of the FP-operator Mp. The latter has a one-dimensional 
eigenspace corresponding to the eigenvalue 1, since the kernel V{x,y) vanishes only for x — y — 1/2 (mod 1), which 
implies that the two-step probability / 'P{x, z)'P{z,y)dz > for x,y G X (see Q, Th. 5.7.4). Thus the equation ( |30| ) 
has the unique solution q{P) fulfilling 
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E9(^)r / My)dy^i, (34) 



r=0 



or equivalently {q{P) , t) = 1 , where t = {Jx ^oiy)dy, Jx {y)dy, ■ ■ ■ , Jx {y)dy) ■ We find it diagonalizing numeri- 
cally the matrix D. The function pp given by (Gfl) is then the invariant density for the system with noise fa- 

This technique was used to find the invariant measure for the logistic map given by f{x) = 4a;(l — x), for x G X in 
the presence of noise. Fig. ^ presents the invariant density for the logistic map with noise parameter TV — 20. It can 
be compared to the histogram showing the density of the 11-th iteration of one million imiformly distributed random 
initial points. 

In the deterministic limit a ^ the size of the matrix TV -I- 1 grows to infinity. We believe that our approach can 
be used to approximate the invariant measure of the deterministic system by decreasing the noise strength. Fig. ^ 
presents a plot of invariant densities for the logistic map perturbed with the trigonometric noise for N = 10, 20, 50 
and 100, compared with the invariant measure for the deterministic case TV — > oo given by |42[| 



piy) - / ' , (35) 



for y G X. 



C. Spectrum of randomly perturbed systems 

The spectral decomposition of the Frobenuis-Perron operators corresponding to classical maps is a subject of an 
intensive current research ||4^- p7| , [l7| , ^ , ^ . The spectrum of a FP-operator is contained in the unit disk on the 
complex plane and depends on the choice of a function space, in which acts the FP-operator. If the dynamical 
system has an invariant density exists, the largest eigenvalue is equal to the unity. The radius of the second largest 
eigenvalue determines the rate of convergence to the invariant measure. To characterize the spectrum one defines 
essential spectral radius r. It is the smallest non-negative number, for which the elements of the spectrum outside 
the disk of radius r, centered at the origin, are isolated eigenvalues of finite multiplicity. It was shown p5| that for 
one-dimensional piecewise expanding maps and the FP-operator defined on the space of functions of bounded 
variations, the spectral radius is related to the expanding constant. 

We analyzed the spectral properties of the FP-operator of the perturbation of the logistic map, for which the 
Lyapunov exponents equals to ln(2) and r = 1/2. The interval [0,1] is joined into a circle to keep the system 
conservative in the presence of noise. FP-operator of the system subjected to the shift-invariant additive perturbation 
■Pjv is represented by the matrix D of the size TV -I- 1. We obtained its spectrum by the numerical diagonalization. The 
largest eigenvalue Ai of D was equal to the unity up to a numerical error of order 10~^°. The second eigenvalue A2 
was found to approach the essential spectral radius r in the deterministic limit TV — > 00. Since the matrix D has real 
entries, its eigenvalues are real or appear in conjugate pairs. Fig. |5| presents the largest eigenvalues of this system for 
TV = 10, 20, 50 and 100. All other eigenvalues are so small that they coincide with the origin in the picture. Observe, 
that eigenvalues do not tend to the values Am — 1/4™^^ for to = 1, 2, . . . found for the deterministic system in [^ . 

Our results show that the structure of the spectrum of the FP-operator of a deterministic system depends on 
the character of the method used to approximate it. Introducing a random noise may be considered as a possible 
approximation, since it enables us to represent the FP-operator by a matrix of a finite dimension. In other words, the 
presence of the noise predetermines a certain space, in which the eigenstates live. This numerical finding corresponds 
to the recent results of Blank and Keller Q, who showed the instability of the spectrum for some maps subjected to 
certain perturbations. 
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IV. COMPUTING ENTROPY FOR SYSTEMS WITH EXPANDABLE NOISE 
A. Matrix formulation of probability integrals 

Our aim is to compute probabilities entering the definition of the total dynamical entropy of a noisy system Pio,...,i„_i 

-Pio,...,i„-l = / Pp{xo)dxo / dxi... / dXn-l'P{f{xo),Xi)'P{f{xi),X2)---'P{f{Xn-2),Xn-l). (36) 

Introducing (n — 1) times the expansion ( pO[ ) applied to the kernel 'P{f{x), y) and interchanging the order of summing 
and integration we arrive at 

^^o,■■■.^,.-^ = [D{tn-l) ' ' ' D{t,)D{io)] q{P) , (37) 

where A, r and q{P) are defined in Sect. IIIB, D{i) — B{i)A, and matrices B{i) are given by the integral over the 
cell Ei, i.e. B{i)ri = J^Ur{f{x))vi{x)dx ior i — 1, . . . , k; r, I = 0, . . . , N in the analogy to (|3l|). 

The above formula provides a significant simplification in the computation of entropy. Instead of performing 
multidimensional integrals in (^6|), we start from computing the matrices D{i) for any cell i = 1, . . . , k, and receive 
the desired probabilities by matrix multiplications. By this method the probabilities may be efRciently obtained even 
for larger numbers of the time steps n. The only problem consists in the number of terms in (^), equal to A;", which 
for larger number of cells k becomes prohibitively high. To overcome this difficulty we apply in this case the technique 
of iterated functions systems presented below. 



B. Computation of entropy via IFS 

In this section we present a method of computing the dynamical entropy (0), which is especially useful when the 
number of cells k of the partition of the space X is large. We use the concept of iterated function systems (IFSs), 
discussed in details in the book of Barnsley . Consider the set of k functions pi :Y M+ and maps Fi : Y i-^ Y 
defined as |50|j5l| 

D{i)z t^l, ...,k,zeY, (38) 

- 1^ 
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where the vector r and the matrices D{i) are defined in Sects. IIIB and IVA, respectively, and Y C 
closure of the set of all vectors of the form u{f{x)) for x G [0, 1]. 

Let us stress that the spaces X = [0, 1] and the 1 dimensional space Y are different. The normalization of the 
kernel J-^V{f{x),y)dy — 1 ioi x G X, leads to the condition J2i=iPii^) — 1 ^^r any z eY. Therefore the functions 
Pi can be interpreted as place dependent probabilities and together with the functions Fi form an IFS. It is uniquely 
determined by the dynamical system / with the noise given by the density V and a specific fc-elements partition 
A. Thus, the number of cells k determines the size of IFS. It can be shown that the entropy of the considered 
dynamical system with noise is equal to the entropy of the associated IFS. 

The IFS generates a Markov operator Ai acting on the space of all probability measures on Y. For any measurable 
set S CY the following equality holds 

{Miy){S) = V / p,iw)di^iw). (39) 

It describes the evolution of the measure v transformed by A^. If the functions Fi fulfill the strong contraction 
conditions ]49| ], there exists a unique attracting invariant measure /i for this IFS 

Mfi^fi, (40) 

which, in general, displays multifractal properties p^ . The total entropy can be computed as the Shannon entropy 
hk{pi, ■ ■ ■ ,Pk) — — X^iLi Pi l^Pi averaged over the invariant measure |p3| , ^ 

HtotifalA) ^ J^hkipiiy), . . . ,Pk{y))dKy) ■ (41) 

The calculation of such an integral from the definition corresponds to the matrix method presented previously. 
However, the existence of the attracting invariant measure /i and the Kaijser-Elton ergodic theorem ||5^ , |5^ assures 
that 

^ n — 1 

Htot{fa;A)^ lim - Y^hkiyi), (42) 

where {yi\ is a generic random sequence produced by the IFS. Such a method of computing of an integral is often 
called random iterated algorithm W9|. We start computations from an arbitrary initial point yoi iterate it by the IFS, 
and compute the average (^) along a random trajectory. To avoid transient dependence on the initial point yo one 
should not take into account a certain number of initial iterations. Note that the computing time grows only linearly 
with the number of cells k and one does not need to perform the burdensome time limit (^. 

We used a similar method to compute the quantum coherent states entropy psf and the Renyi entropies for certain 
classical deterministic maps |Q. 

V. DYNAMICAL ENTROPY FOR NOISY SYSTEMS - EXEMPLARY RESULTS 

In this section we will study the entropy of the Renyi map and the logistic map perturbed by the trigonometric 
noise given by (pT]). We will consider the partitions Ak of the interval [0,1] into k equal subintervals. We put 
H{k) H{Ak), Htot{N,k) Htot{fN]Ak), Hnotse{N,k) := ff„ozse(/Ar; A), Hdyn{N,k) := Hdyn{fN]Ak) and 

Hdyn{N) := HdynifN)- 

A. Boltzmann— Gibbs entropy 

A simple integration allows us to obtain the BG-entropy Hbg{N) for this kind of the noise 



Hbg{N)^- / deCNCOs~(^e)ln[CNCOS^«)] =^ [*(f)-*(^)] +l-lna 



(43) 



where ^ denotes the digamma function |Q and the normalization constant Cn is given by (^2|) . 

It follows from (E^) that in the deterministic limit {N — > c>o) the BG-entropy diverges to minus infinity, namely 
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lim-^-^^i. (44) 
AT^oo In AT 2 ^ ' 

This relation shows, how the maximal dynamical entropy ~Hbg (see ([l8|)), admissible by a certain level of the noise, 
grows logarithmically in the deterministic limit. 



B. Entropy of the noise 



We used the matrix method of computing probabilities, which lead to partial entropies 77„ and the relative entropies 
Gn- Fast (presumably exponential) convergence of the sequence Gn allowed us to approximate the entropy by Gj 
with accuracy of order « 10~®. Fig. |^ presents the dependence of the entropy of the noise Hnoise{N, k) on the number 
of cells k in the partition Ak for two different amplitudes of noise {N — 10 and 20). The data for large number of cells 
{k > 20) are obtained by the technique of IFS. The results are compared with the upper and lower bounds for the 
entropy of the noise which occurred in (^^. It follows from (14) that the entropy diverges logarithmically with the 
number of cells k in the partition. For a fixed partition it decreases to zero with decreasing strength of the stochastic 
perturbation (increasing parameter N). 




C. Entropy for the noisy Renyi map 



The Renyi map f(s)i^) — [s2;]modi (s G N), with explicitly known metric entropy HKsif{s)) — Ins, is particularly 
suitable to test changes of the dynamical entropy with stochastic perturbation. Results obtained for the trigonometric 
noise (|2l] ) are much more accurate than these obtained for rectangular noise and reviewed briefly in Sec. IID. Data 
presented below are received for the Renyi map with s = 6 (we put / = /(e))- Dependence of the total entropy 
Htot{N,k) on the number of cells k is presented in Fig. 0a for four levels of noise {N = 10,20,50 and 100). The 
solid line represents the entropy of the partition H{k) — Ink (upper bound) and the dashed line provides the N- 
dependent lower bound given by H{k) + Hbg{N) (for N = 10), while the stars denote the partition dependent 
entropy of the deterministic system given by /. It saturates at the generating partition kg = 6 and achieves the value 
Hxsif) = ^'^(6). It seems that this value gives an another lower bound for the total entropy Htot- 

The total entropy and the entropy of the noise diverge in the limit of fine partition J, (/c — > oo), but their 
difference remains bounded. 
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FIG. 7. Entropies for the Renyi map f{x) = [6a;]niod i perturbed by the noise with A'' = 10(o), 20(Q), 50(+) and 100(A) as a 
function of the number of cells k: a) The total entropy Htotik). Solid line represents the upper bound {Ha,.) while the dashed 
line provides the lower bound (|l|) for iV = 10. b) The dynamical entropy Hdy„{k). The maximum of each curve gives Hdyn 
as represented on the right side. 



H„ 



Fig. 0b shows the difference Hdyn{N,k) = Htot{N,k) 
entropy (^. This quantity tends to zero for k = 1 and k - 
for the dynamical entropy H(iyn{N) - close to the number of cells k 



[N, k) necessary for computation the dynamical 
oo (17) and achieves its maximum - giving a lower bound 
in the generating partition. Dynamical entropy 
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is equal to zero for = and increases with the decreasing noise strength. In the hmit ^ cxd it is conjectured to 
tend to the KS-entropy of the deterministic system Hks = ln(6) represented by a horizontal line. 



D. Entropy for the noisy logistic map 



A similar study was performed for the logistic map given by f{x) = 'ix{l — x) for x G [0, 1] perturbed by the 
trigonometric noise (|2l|). As before we treat the interval X as a circle setting f{x) — /(xmodi)- Numerical data 
produce pictures analogous to those obtained for the Renyi map with s = 6. Instead of presenting them here, we 
supply a compilation of the results for both systems. Computing total entropy and entropy of the noise for several 
partitions we took the largest difference between them as an approximation of the dynamical entropy . Fig. |^ shows 
how the dynamical entropy changes with the noise parameter N for both systems. It is conjectured to tend to the 
corresponding values of the KS-entropy (ln(2) for the logistic map and ln(6) for the Renyi map) in the deterministic 
limit iV — > oo. 
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FIG. 8. Dynamical entropy Hdyn for the Renyi map (o) and the logistic map (□) depicted as functions of the noise parameter 
A^. Horizontal lines represent the values ln(6) and ln(2) of the KS-entropy for the corresponding deterministic maps. 



VI. CONCLUDING REMARKS 



The standard definition of the Kolmogorov-Sinai entropy is not applicable for systems in the presence of a continuous 
random noise, since the partition dependent entropy diverges in the limit of a fine partition. We generalize the notion 
of the KS-entropy for dynamical systems perturbed by an uncorrelated additive noise. The total entropy of a random 
system is split into two parts: the dynamical entropy and the entropy of the noise. In the deterministic limit (the 
variance of the noise tends to zero) the entropy of the noise vanishes, while the dynamical entropy of the stochastically 
perturbed system is conjectured to tend to the KS-entropy of the deterministic system. 

The continuous Boltzmann-Gibbs entropy characterizes the density of the distribution of the noise. It provides an 
upper bound for the maximal dynamical entropy observable under the presence of this noise. If the BG-entropy is 
equal to zero such a noise may be called disruptive, because one cannot draw out any information concerning the 
underlying deterministic dynamics. Investigating properties of the dynamical entropy we find that the two limits: the 
diameter of the partition to zero and the noise strength to zero do not commute, and point out some consequences of 
this fact. 

Computation of the dynamical entropy becomes easier, if one assumes that the density of the noise can be ex- 
panded in a finite basis consisting of continuous base functions. In this case we find a simple way of computing the 
probabilities of trajectories passing through a given sequence of the cells in the partition. The calculations are based 
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on multiplication of matrices of size N + 1 and the computing time grows linearly with the length of a trajectory n. 
On the other hand, diminishing the noise strength causes an increase of the matrix dimension. 

For each dynamical system perturbed by this kind of noise and for a given /c-element partition of the phase space 
we construct an associated iterated function system, which consists of k functions with place dependent probabilities 
and acts in a certain A'^ + 1 dimensional auxiliary space. Entropy of the dynamical system with noise is shown to 
be equal to the entropy of IFS, which can be easily computed by the random iterated algorithm. This method is 
particularly suitable for large number of cells fc, for which the number of possible trajectories grows in time as fc". 

We study some one-dimensional maps perturbed by trigonometric noise, for which the basis functions are given by 
trigonometric functions. In this case we can represent the Frobenius-Perron operator for the noisy system by a matrix 
of size + 1. Diagonalizing this matrix numerically we find the spectrum of this operator. Analysing the logistic 
map subjected by such a random perturbation we indicate that the invariant measure tends to the invariant measure 
of the deterministic system in the limit N — > oo. On the other hand, the spectrum of the Frobenius-Perron operator 
describing the noisy system need not to tend to the corresponding characteristics of the deterministic system. 

The deterministic limit N —^ oo resembles in a sense the semiclassical limit of quantum mechanics h ~* 0. For 
example, if one discuss the quantum analogues of classical maps on the sphere the size of the Hilbert space 
2j + 1 behaves as 1/h, where j is the spin quantum number. Therefore, it would be interesting to analyze such 
two-dimensional classical systems in the presence of noise (in the case of two-dimensional "trigonometric" noise the 
FP-operator can be represented by a matrix of the size N'^) and to compare, how the spectrum of a given classical map 
is approached in two complementary limits: the semiclassical limit j — > oo of the corresponding quantum map and 
the deterministic limit A'^ — > oo of a classical noisy system. Some preliminary results on related issue of truncatin g th e 
infinite matrix which represents the FP-operator of a deterministic system have been achieved very recently p8|p9t . 

One of us (KZ) would like to thank Ed Ott for the hospitality at the University of Maryland, where a part of 
this work has been done and gratefully acknowledges the Fulbright Fellowship. This work was also supported by the 
Pohsh KBN grant P03B 060 13. 



APPENDIX A: EXEMPLARY ITERATED FUNCTION SYSTEM 

To illustrate the IFS method we discuss the computation of the entropy of the noise given by (|l]) for AT = 2 and for 
the partition of the interval [0, 1] into k — 4 equal cells. In this case gives the IFS consisting of fc = 4 functions 
acting in a 3-dimensional space Y C [—1, 1]'^. The probabilities pi are place dependent 



/ N x(Tr+2) . y I z(Tr-2) / n x(Tr-2) , y , z(7r+2) 



while the functions read 
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for [x, H, z) G y. 

Fig. presents the support of the invariant measure /i for this IFS. Applying the random iteration algorithm we 
obtain in this case the entropy of the noise Hnoise — 1.1934. Ironically, less interesting (more contracting) fractal 
picture leads to a faster convergence of the sum (Ea). 
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2 -2 

FIG. 9. Fractal support of the invariant nicasurc i^* of the IFS associated with the trivial dynamical system f{x) = x in the 
presence of the trigonometric noise with N = 2. The number of cells fe = 4 determines the number of functions in the IFS and 
the structure of the depicted set. 
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